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Abstract 

In this work, a new lower bound for the maximal error probability of 
a two-user discrete memoryless (DM) multiple-access channel (MAC) is 
derived. This is the first bound of this type that explicitly imposes inde- 
pendence of the users' input distributions (conditioned on the time-sharing 
auxiliary variable) and thus results in a tighter sphere-packing exponent 
when compared to the tightest known exponent derived by Haroutunian. 

1 introduction 

An interesting problem in network information theory is to determine the mini- 
mum probability of error which can be achieved on a discrete memoryless (DM) , 
multiple-access channel (MAC). More specifically, a two-user DM-MAC is de- 
fined by a stochastic matrb{3 W : X x y — > Z, where the input alphabets, X,y, 
and the output alphabet, Z, arc finite sets. The channel transition probability 
for sequences of length n is given by 

n 

W n (z\^,y)^l[W(z i \x i ,y i ) (1) 
i=l 

where 

x ^(x u ...,x n )eX n ,y^( yi ,...,y n )ey n 

and 



1 Wc use the following notation throughout this work. Script capitals U, X, y, Z,. . . denote 
finite, nonempty sets. To show the cardinality of a set X, we use \X\. We also use the letters 
P, Q,. . . for probability distributions on finite sets, and U, X, Y ,. . . for random variables. 
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It is known [1], that for any (Rx,Ry) in the interior of a certain set C, and for all 
sufficiently large n, there exists a multiuser code with an arbitrary small average 
probability of error. Conversely, for any (Rx,Ry) outside of C. the average 
probability of error is bounded away from 0. The set C, which is called capacity 
region for W, is the closure of the set of all rate pairs (Rx,Ry) satisfying [2] 

0<Rx <I(XAZ\Y,Q) (2a) 
0<Ry <I(YAZ\X,Q) (2b) 
0<Rx+Ry <I(XYAZ\Q), (2c) 

for all choices of joint distributions over the random variables Q, X, Y, Z of 
the form P(q)P(x\q)P(y\q)W{z\x, y) with Q g Q and \Q\ < 4. 

Haroutunian [3] derived a lower bound on the optimal average error proba- 
bility for W . This result asserts that the average probability of error is bounded 
below by exp{— nE sp (Rx, Ry , W)}, where 

E sp (R x ,R Yl W) 4 max min D{V z \ xy \\W\Pxy). (3) 

PxY V Z \XY 

Here, the maximum is taken over all possible joint distributions over the random 
variables X, Y, and the minimum over all test channels Vz\xy which satisfy at 
least one of the following conditions 

I V (XAZ\Y) < R x (4a) 
I V (Y AZ\X) < R Y (4b) 
I V {XYAZ) <R X + Ry, (4c) 

where V = Vz\xy x P.xy- This bound tends to be somewhat loose because it 
does not take into account the separation of the two encoders in the MAC. 

In this paper, we derive a new lower bound that explicitly captures the 
separation of the encoders in the MAC and thus is tighter than the one provided 
by Haroutunian. However, this bound is only valid for the maximal and not the 
average error probability. Nevertheless, we believe that the techniques used in 
this derivation can be extended to provide lower bounds for the average error 
probability as well. 

The paper is organized as follows. First, some preliminaries are introduced 
in section [2j Then in section [3l we state and prove the main result. The proof 
hinges upon a strong converse theorem which is also stated in the same Section 
and proved in the Appendix. 



2 Preliminaries 

For any alphabet X, V(X) denotes the set of all probability distributions on 
X. The type of a sequence x = [x\, ...,x n ) g X n is the distributions P x on X 
defined by 

P x ( x ) A Ijv(x|x), x £ X, (5) 
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where A(x|x) denotes the number of occurrences of x in x. Let V n (X) denote 
the set of all types in X n , and define the set of all sequences in X n of type P as 

T P ^{xe X n : P x = P}. (6) 

The joint type of a pair (x, y) £ X n x y n is the probability distribution P X;y 
on X x y defined by 

P x , y (x, y) 4 1n(x, y|x, y), (x, y)£Xxy, (7) 

where JV(x,y|x, y) is the number of occurrences of (x,y) in (x, y). The rela- 
tive entropy or Kullback Leibler distance between two probability distribution 
P, Q £ V{X) is defined as 

£(P||Q)^5>(x)log^. (8) 

Let W(3^|<-f) denote the set of all stochastic matrices with input alphabet X 
and output alphabet y. Then, given stochastic matrices V, W £ W(y\X), the 
conditional I- divergence is defined by 

D(V\\W\P) 4 J2 P(x)D(V(-\x)\\W(-\x)). (9) 

An (n, M, A) code for W : X -> Z, is a system {(u*, A) : 1 < i < M} with 

• Ui G X n , Di C Z n 

• Di n A' = for i 7^ t' 

• r(A|u,-)>l-A, forl<i<M. 

An (n, M, N) multi-user code is a set {(u it Vj, Aj) : 1 < « < M, 1 < j < N} 
with 

• Uj G X n , Vj £ y n , Du C z n 

• D jj n%=0for(i,i)^(i',/). 

Finally, an (n, M, iV, A) code for the MAC, W, is an (n, M, AT) code with 

M JV 

-^^^(5,^,^) > i — a. (io) 

i=i j=i 

3 main result 

The main result of this paper is a lower (sphere packing) bound for the maximal 
error probability for a MAC. To state the new bound we need an intermediate 
result that has the form of a strong converse for the MAC. We state this result 
here and relegate the proof to the appendix. 
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Definition 1. For any DM-MAC, W , for any joint distribution P G V{X x y), 
any < A < 1, and any (n, M, N) code, C ', define 

8 W (P,\) 4 {( Ui , Vj -) e C : W(Ai|ui, Vj -) > i^, 

(«i,Vj)6T P }. (11) 

Theorem 1. Consider any (n,M,N) code C. For every P XY € "Pn[X x 30, 
such that \S W (P% Y ,\)\ > ( „ +1) Vnvi (1 - TfxWN, then 

(ilogM,ilogAr)GC^(P^) (12) 

where C^(P) is defined as the closure of the set of all (Ri, R2) pairs satisfying 

Ri<I{X AZ\Y,Q) + e n (13a) 
i? 2 </(F AZ|X,Q) + e„ (13b) 
i?i+i?2 < I(XY A Z\Q) + e n (13c) 

/or some choice of random variables Q defined on {1,2,3,4}, and joint distri- 
bution p(q)p(x\q)p(y\q)w(z\x,y), with marginal distribution p(x,y) = P n (x,y). 
Here, e n — ► anre-tco. 

We further define Cw(P) (the limiting version of the sets C{y(P)) as the 
closure of the set of all (i?i, R2) pairs satisfying 

Ri < I(X A Z\Y, Q) (14a) 
R 2 <I(YAZ\X,Q) (14b) 
R1+R2 <I{XYAZ\Q), (14c) 

for some choice of random variables Q defined on {1, 2, 3, 4}, and joint distribu- 
tion p(q)p(x\q)p(y\q)w(z\x, y), with marginal distribution p(x, y) = P(x, y). 

Theorem 2. (Sphere Packing Bound). For any Rx,Ry > 0, <5 > and any 
DM-MAC, W :X -> Z, every (n, M, N, A) code, C with 

-log A/ > R x +5 (15a) 
n 

-log TV > R Y +5, (15b) 
n 

has maximum probability of error 

P? >^exp(-nE sp (R x ,R Y ,W)(l + S)), (16) 

where 

E sp (R x ,R Y ,W)± max min D(V\\W\P X y)- (17) 

P XY £V(Xxy)V:(R x ,R Y ) 
<£Cv(P.xy) 



4 



Proof. If A = 1, the result is trivial. Assume A < 1. Let's choose A' such 
that max{l — 5, A} < A' < 1. Since A' > A, every (n, M, N, A) code is also an 
(n,M,N,X') code. Using the same argument as [4, pp. 189], we conclude that 
there exist at least one dominant type P n € V n (Xxy), such that \£w(P n ,A')\ > 



(1 - j^rj)MN. Consider an arbitrary DM-MAC V : X x y 



such that (R x ,Ry) i C^{P n ). By Theorem Q] there exist at least one pair 
(ui, Vj) with joint type P X y such that 



V n (DC j \u i ,v j )>^>l--. (18) 



„/ ,, x r D(V\\W\P n ) + h(l - |)i 



1 + A' 1 8 

Using the same method as Csiszar in [5, pp. 167], we have 

D(V\\W\P n ) + h(l - |) 
~§_ 

2 

>±exp{-nD(V\\W\P n )(l + S)}, (19) 

for small enough 8 satisfying h(l — |) < 1— |. By maximizing the result over 
the arbitrary channel V , we get 

P e m > max -exp{ - nD(V\\W\P n ){l + 6)} 

V:(Rx -.Ry) 2 

= \ exp { - n min D(V||W|P»)(1 + 5)} 

> mm -exp{-n min D{V\\W\P n ){\ + 8)} 

P n e 2 1 V:(R x ,Ry) 

r„(xxy) £C£(p") 

> mm -exp{-n min D{V\\W\P){1 + 8)} (20) 
v(xxy) iCv(P) 

Using Lemma [31 we conclude that for sufficiently large n, 

min — exp {-n min L>(U| |W|P)(1 + 8)} . (21) 



fG 2 V:(R x ,Ry) 



□ 



which completes the proof. 



4 appendix 

The basic idea of the proof is wringing technique which was used for the first 
time, by Ahlswede [6]. 
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Consider any P% Y e V n {X x y), such that |£ W (P£ F ,A)| > (ro+1) Vi|y| (1 - 

^)MiV. let's define ^ 4 : W(A>i, v,-) > ^.kvj) 6 T P » y }. 

Since |*4| = |£y^(PJ r , A)|, we conclude that 

'^^(^TT)™ (1 -TTa )M7V - (22) 

Define, 

C(i) = {(i J i):(»,i)GAl<J<^} (23a) 

B(j) = : e A 1 < i < M}. (23b) 

Consider the subcode {(iij, Vj, -D^) : € .4} and define random variables 

Fr((X",r") = (ui.vj)) = |l|if G A (24) 

Lemma 1. For random variables X n , Y n defined in (|24|) , </ie mutual informa- 
tion satisfies the following inequality: 

2A 

/(X" A y n ) < — log(l - — ) + \X\\y\ log(n + 1). (25) 
Proof. This is a generalization of the proof by Dueck in [4] . Observe that 

H(Y n \X n ) = Pr(X n = Ui)H(Y n \ui). (26) 

However, by the definition of the variables X n , Y n we have 

H(Y n \u i )=log\{j:(i,j)eA}\ (27) 



and 



Hence, 



Pr(X n = u i ) = \A\- 1 .\{j:(i,j)€A}\. (28) 



H(Y n \X r 



M 



= I-^EKj ■ &3) e ^}|log|{j : e (29) 

i=l 

In the right hand side of (|29|). the summands are of the form mlogm. This 
function of m is increasing and convex in m. Thus, 

H(Y n \X n ) > 

M M 

\{J ■ (id) e -4} |) MM- 1 ^ |{j : (i, j) €A}\), (30) 
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and since 

M 



52\{j:(i,j)eA}\ = \A\, (31) 
j=i 

we have 

> log(A/-VD- (32) 

By (f22j), we conclude that 

F(y n |x") > 

2A 

logiV + log(l - — ) - log(n + 1). (33) 

Finally, 

I(X n A F") = H(Y n ) - H(Y n \X n ) 

< log N-H(Y n \X n ), (34) 

which concludes the proof. □ 

Lemma 2. [7] Let X n , Y n be RV's with values in X n , y n resp. and assume 
that 

I(X n AY n )<a (35) 

Then, for any < 5 < a there exist ti,t2, —,tk € {1, ■■■,n} where < k < -f 
such that for some x tl , y tl , x t2 , y~t 2 , ■■■,Xt h ,Vt k 

I(X t AY t \X tl =x tl ,Y tl = y tl ,...,X tk = x tk ,Y tk = y t J <6 

fort = 1,2, ...,n (36) 

and 

Pr(X tl =x tl ,Y tl =y tl ,...,X tk = x tk ,Y tk = y tk ) 

~ { \X\\y\{2a-5) ] ■ (3?) 
Consider the subcode {(u^, v_,, D^) : (i,j) £ A}, where 

A — {{i,j) £ A : Ui t[ ~ x t[ ,\rj t[ = y t[ 1 < I < k} (38) 

and define 

C(i) = {(i,j):(i,j)eA,l<j<N} (39a) 

B(j) = : d,l<i< M}. (39b) 
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Lemma 3. The subcode {(uj,Vj, Dij) : G -4}, is a subcode with maximal 
error probability -4p , and 

H S ' m »-D ^' <40) 

Moreover, 

\Pr{X t = x, Y t = y)- Pr(X t = x)Pr(Y t = y)\ < 25 1 ' 2 , (41) 

where X n = (Xi, ...,X n ), Y" = (Yi,...,Y n ) are distributed according to the 
Fano- distribution of the subcode {(u,, Vj, Dij) : (i,j) € A}. 

Proof. Since A C A, the maximal probability of error for this subcode is at 
most The second part of Lemma [2 yields immediately (|40|) . On the other 
hand, 

P A (X t =x,Y t = y\x tl ,y tl ,x t2 ,yt 2 ,-,xt k ,yt k ) 
= PAXt = x,Y t = y,x tl ,y tl ,xt 2 ,yt 2 , ...,x tk ,y tk ) 
■fU(*ti , Vti ,x t3 ,yt 3 ,—, x tk , y tk ) 
N A (X t =x,Y t = y,x tl ,y tl ,x t2 ,yt 2 , ...,x tk ,y tk ) 



NA{x tl ,y tl ,xt 2 ,yt 2 , ...,x tk ,y tk ) 
N^Xt = x^Yt = y) 
\A\ 

= P A (X t =x,Y t =y), (42) 
Therefore, by the first part of Lemma [21 we conclude that 

I(X t A Y t ) < 5, for 1 < t < n. (43) 
Since I(X t A Y t ) is an I-divergence, Pinskcr's inequality implies [8] 

^ \Pr{X t =x,Y t =y)- Pr{X t = x)Pr(Y t = y)\ < 25 1/2 . (44) 



□ 



Lemma 4. [9]: For a (n,M,X) code {(ui,Di) : 1 < i < M} for the non- 
stationary DMC (Wt)tS=i 

n „ 

logM< ^J(X t AZ t ) + ——\X\n 1 / 2 , (45) 
t=i 

where the distribution of the RV's are determined by the Fano- distribution on 
the codewords. 
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Define random variables X n , Y n on X n resp. y n by 

1 



Pr((X n ,Y n ) = ( Ul , Vj )) = —if € A (46) 



Lemma 5. For any < A < 1, any (n, M, AT) code C = {; u,. v ,./),, : : 1 < i < 

M, 1 < j < N} for the any MAC, W , and for any Pg Y G P n (X X y), such that 
\£w{P^ Y A)\ > (w+1) Wl ft ~ ITa) M ^ 

logM < 5^/(X t A Z t |Y t ) +c 1 (A)n 1 / 2 + Cl fclog(— ) 

t=i 

logiV < ^/(Y A Z t |X t ) +c 2 (A)n 1 / 2 + c 2 fclog(— ) 
t=i 

" 2er 
logMiV < A Zt) + c 3 (A)n 1 / 2 + c 3 fclog( — ), 

where the distributions of the RV's are determined by the Fano- distribution on 
the codewords {(uj,Vj) : € -A}- Here, Cj(A) and Cj are suitable functions 

ofX. 

Proof. For any fixed j, consider (n, |<B(j)|) code {(Uj,-Dy-) : € Any 
pair of codewords in this code has probability of error at most equal to ^fe^- 
Let's define A' = It follows from Lemma 2] that 

n „ 
log|B(j)| < Y,I(Xt A Z t |F t = v jt ) + T -^|A'|, 1 1 / 2 . (47) 
t=i 

Similarly, 

n „ 
log|C(i)| < ^I(F 4 A Zt|X t = u«) + Y^l^l" 172 (48) 
t=i 

n „ 

log |^| < Y^HXtYt A Z t ) + Y^IA-I^ln 1 / 2 . (49) 
t=i 

Since Pr(F 4 = y) = l^ 1 E (i>i) ^ Wv}, 
l^l" 1 £ Iog|B(j)l 

< E E^a^i^v^ 



Define A* = 



A 2A 
1 + A 



Therefore, 



and 



B* ^ 



1 - A* 



M 



n (n+ 1)1*11*1 y \X\\y\{2a- S) 



lAr 1 y lo si^')i 

= |^|- 1 ^|B(i)|log|B(i)| 



^ia]- 1 y mmog\B(j)\ 

j:\B{j)\>B* 

^l^l^logCB*) y toi 
r-\B(j)\>B* 

> |^|- 1 log(B*)(|^| -NB*). 
By lemma H (H2J), and definition of B* , 



NB* < -\A\ 



Therefore, 



iai- 1 y l °s\B(j)\ 

(ij)e-4 



1 



>|^r 1 log( J B*)(|^|--|^|) 
n 

1 1-A* M 
= (1- -)log( 



;)*)• 



By O, JSU 



n (n+l)l*H3'l v '|Ar||y|(2tr-d) 



logM < (1 + A Z t |f t ) + — ^lAfJn 1 / 2 ) 

n ' 1 — A' 

t=i 

- log(l - A*) + logn + |#||y| log(n + 1) 



< ]T J(X t A Z t |? t ) + Ci (A')ti 1/2 + dHog(^) 
t=l 
+ 2|Z| 



Analogously, 



log TV < ^I(y t A Z t |X t ) + ^(A'Jn 1 / 2 + c 2 fclog(^) 



2\Z\. 
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To find an upper bound for log MN, we first try to find a lower bound on the 
log | A | . By Lemma [3] 

log \A\> log \A\ + Hog( M ^__ ) 

^ 1 ° g| - 4|+fcl0g( LYT^ ) 

= log |^|-fclog(y)-*log(|Af| |y|) 

2A 

> log(MiV) - \X\\y\ \og(n + 1) + log(l - — — ) 

1 + A 

-fciog(y)-*iog(|Ar||y|). (57) 



Therefore, 



Using gl]), 



log(MiV) < log |^| +c 3 fclog(y). (58) 



2(7 

logAfiV <Y^I{X t Y t A Z t ) + c 3 (A')n 1 / 2 + c 3 fclog( — ). (59) 

□ 

Note that, in general X t and Y t are not independent. In the following, we 
prove that they are nearly independent. 

Now, we combine (f2~5"j) and lemma [3] For an (n,M,N) code {(u;, Vj, Dij) : 
1 < i < M, 1 < j < N} which has the particular property mentioned in 
theorem [I] define A, A as defined before. Apply lemma [3] with parameter 
5 = -nr 1 !" 1 . Using a = -log(l - ^) + \X\\y\ log(n + 1), we conclude that 

* < y = 2Vn(- log(l - ^) + \X\\y\ log(n + 1)) 
-O(V^logn) (60) 

and 

\Pr(X t =x,Y t =y)- Pr(X t = x)Pr{Y t = y)\ < 2n~ l / A , (61) 

for any x G X, y € y, and t = 1, ...,n. We can write the above equations as 
follows 

- log M <-V 7(X t A Z t \Y t ) + C(A)^ (62a) 
n n * — ' n 

t=i 

-logN<-yi(Y t AZ t \X t ) + C(X)^- (62b) 
n n n 

— log MN < -Y I{X t Y t AZ f ) + C(A)^. (62c) 
t=i 
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The expressions in (|62ap - (|62c[) are the averages of the mutual informations 
calculated at the empirical distributions in the column t of the mentioned 
subcode. We can rewrite these equations with the new variable Q, where 
Q = q G {1,2, ...,n} with probability — . Using the same method as Cover 
[1. pp. 402], we obtain the result. The only thing remained to be found is the 
distribution under which wc calculate the mutual informations. However, by 

m 

\P(X = x,Y = y\Q = q)- P(X = x\Q = q)P(Y = y\Q = q)\ 
= \Pr{X q = x, Y q = y)- Pr(X q = x)Pr(Y q = y)\ 
< 2n~ 1 l i . (63) 

Using the continuity of conditional mutual information with respect to distribu- 
tions, using the same idea of [10, pp. 722], we conclude that, if two distributions 
are close, the conditional mutual informations, calculated based on them, can- 
not be too far. More precisely, we can say that there exists a sequence {5 n } n % 1 , 
5 n — * as n — > oo, such that, 

- logM < I(X t A Z t \Y t ,Q) + C(A)^ + S n 
n n 

— — — o(n ) 

- logiV < I(Y t A Z t \X u Q) + C(A)-^ + S n 
n n 

- log MN < I(X t Y t AZ tl Q) + C(A)^ + S n . (64) 
n n 

Here, the mutual informations calculated based on p{q)p{x\q)p(y\q)w(z\x,y), 
with marginal distribution P^ y (x,y). On the other hand, the joint probability 
distribution of X and Y is 

P{X = xS = y) 

= P(X(W 1 ) = x,Y(W 2 ) = y,W 1 =i,W 2 = J ) 

= P(X(i)=x,Y(j)=y)P(i,j) 

= m E P(X(i) = x,Y(j) = y) 

1 1 (hj)eA 

n 

= TT\ £ -52l{X q (i)=x,Y q (j)=y} (65) 
However, all codewords have the same joint type PJy, therefore, 

n 

^ l{X q {i) = x,Y q (j) = y} = nPZ Y ( Xl y). (66) 

9=1 
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(165)) and (|60|) result in 

P(X = x,Y = y) = P%y(x,y). (67) 

Finally, we can conclude that 

P(q, x, y, z) = p(q)p(x\q)p(y\q)W(z\x, y), (68) 

in which the marginal distribution of X and Y is PJy(x,?/). 

The cardinality bound on the time-sharing random variable, Q, is the con- 
sequence of Carathcodory's theorem on the convex set [11], [12], [1]. 

Lemma 6. For any fixed P G V{X x y), rate pair (Rx, Ry), 

lim min D(V\\W\P) = min D(V\\W\P), (69) 

n-»oo VED n {P) VeD(P) 

where 

D(P) 4 {v. (R x ,Ry) £C v {P)} 

D n (P) ^ {V : {Rx,Ry) £ Cy(P)}. (70) 

Proof. Define a n = min yeD „ (P) D(y||W|P), and a* = minyg^p) D(V\\W\P). 
Moreover, suppose a* is achieved by V*. Since {a n }^ =1 is a decreasing sequence 
and it is bounded from below (a n > a*), therefore it has a limit. Suppose the 
limit is not equal to a*. Therefore, there exist a 6 > 0, such that for all 
sufficiently large n, 

K-a*|><5. (71) 
Hence, for all 1/ S D n (P), for all sufficiently large n, 

D(V||W|P)-a* >5 (72) 

which concludes that V* cannot belong to D n (P), i.e., for all sufficiently large 

n, 

(Rx,Ry)£0},(P). (73) 

Since V* G D(P), 

(R x ,R Y )$Cv*(P). (74) 
Therefore Cy,(P) cannot converge to Cy(P), which is a contradiction. □ 
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